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(54) Interactive learning systems 

(57) A computer method of automatically providing 
a paperless, individualized, teaming sequence includes 
the step of storing in a computer-readable database a 
plurality of questions directed toward assessing an indi- 
vidual's understanding of a selected learning objective 
for each member of a plurality of different leaming ob- 
jectives. Next, the individual is automatically evaluated 
by randomly and sequentially selecting from the data- 
base at least one question associated with each mem- 
ber of the plurality of leaming objectives and displaying 
each selected question to the individual for a response, 
wherein each response is received and evaluated rela- 
tive to a predetermined standard before the next ques- 
tion is selected (step 122). Next, at least some of the 
results of the evaluations are accumulated in a compu- 
ter storage unit in computer-readable form (step 124) 
and the results processed to automatically produce a 
specific sequence of instructional units in response to 
those responses which varied materially from the re- 
spective standard (step 126). Finally, the sequence of 
instructional units are presented on a display for the in- 
dividual (step 130) wherein the individual can interact 
with each of the units via the display 
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Description 

Computer-based interactive learning systems are known. One particular example is described in U.S. Patent No. 
' 5.395,243 to Lubin et al. entitled Interactive Learning System and which is Assignee to the Assignee hereof. The 
s specification and figures of the Lubin et al. patent are incorporated herein by reference. 

Lubin et al. describes a computer-based interactive leaming system wherein a user can be assisted in learning 
how to use a pre-written application program. Typical application programs include word processing or spread sheet 
operation or operating systems. Lubin et al. describes three different leaming paths. One of the learning paths is a 
preset lesson path in which the user goes through a sequence of audiovisual lessons relating to various predetermined 
10 features of the application program being studied. Upon completion of the lesson sequence, the user is then given an 
opportunity to work with the actual applbation program to re-enforce the previously provided instruction. 

In a second learning path, the user may independently select a topic relating to an aspect of the application program 
which is being studied. The user will then receive audic/vlsual lessons and/or electronically displayed material relating 
to that topic. 

IS In a third leaming path, the user is able to execute the applk:atlon program without receiving any training of the 

type provkled by Lubin's system. During this sequence however, the user may access various help files which can 
present electronic textual reference informatbn relating to that application program while the program is running. This 
process can be carried out without having to exit the applicatbn program. 

The above-described types of learning paths have been found to be useful and effective for many users. Lubin's 

20 system, however, did not incorporate any capability to evaluate a user's level of skill prior to executing any of the 
leaming paths. It wouki be desirable to be able to combine systems of the type taught by Lubin with a fractkxial capability 
to take into account a user's initial level of skill, if any. 

Thus, there continues to be a need for interactive leaming systems which take into account a user's initial skill 
level. It would also be desirable if a user were able to combine lesson elements to create an individualized learning 

25 experience apart from any pre-established learning paths. 

According to an embodiment of the present invention, an Interactive leaming system and method are provkJed 
and can be used for leaming a selected, pre-written program. Preferably, a paperless, individualized, leaming sequence 
can be devetoped and presented to the respective user for the purpose of improving a user's skill and understanding 
with respect to given subject matter. 

30 An apparatus embodying the present inventbn nnay incorporate a prestored, machine-readable database having 

a plurality of questions or exercises which are directed toward assessing the indivkiuals understanding of a selected 
learning objective. Pluralities of questions may be provided for each member of a plurality of different learning objec- 
tives. 

The apparatus may cany out an automatic evaluation of the individual by sequentially selecting from the database 
3S at least one of the questk3ns or exercises associated with each member of the plurality of learning objectives. The 
selected questk>n or exercise may be displayed for the indivbual who in turn enters a response. Each of the responses 
that is received may be evaluated relative to a predetermined standard before the next questbn is selected. 
The system may accumulate the responses and the results of the evaluatbns for subsequent processing. 
The apparatus may in turn produce from the results of the evaluatk)ns a sequence of instructional units, based on 
40 those responses whkih varied materially from the respective standard which are intended to improve the user's skills 
on an individualized basis. The sequence of instructional units can then be presented on a display, for example, for 
the individual. The Indivkjual can then progress through the sequence of units so as to improve selected skills In 
• response to the prior evaluatbn. 

In one embodiment of the present invention, subsequent to completing execution of the sequence of instructional 
4S units, the user can then go through a post-leaming evaluation or test for comparison purposes. 

In yet another embodiment of the present invention, a user can select from anrK>ng a nunnber of object elements 
to Independently create a personalized leaming experience. At a first level, topic^ype object elements can be selected 
which contain, for example, a single objective and an assessment capability At a higher level, lesson-type object 
elements can be selected. The lesson-type object elements incorporate a plurality of related topic-type object elements. 
so In another embodiment of the present inventkjn, at yet another level, unit-type object elements can be selected. The 
unit-type object elements may incorporate a plurality of related tesson-type object elements. 

In yet another embodiment of the present invention, Ieaming4ype objects can be grouped together and associated 
because they incorporate related subject matter or one functions as a prerequisite to another Similarty. unit-type objects 
can in turn be associated with one another because they incorporate related subject matter or because some of them 
ss represent prerequisites for others. 

Reference will now be made, by way of example, to the accompanying drawings, in whk^h: 

Fig. 1 is a bkxk diagram of a system embodying the present inventbn; 
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Fig. 2 is an overall flow diagram of the process of creating an individual learning profile; 

Fig. 3 Is a more detailed flow diagram of one aspect of tfie process of Fig. 2; 

Fig. 4 is a more detailed flow diagram of another aspect of the process of Fig. 2; and 

Fig. 5 is a flow diagram of a process of selecting learning objects for the creation of a personal learning experience. 

5 

While this invention is susceptible of embodiment in many different forms, there are shown in the drawings and 
will be described herein in detail specific embodiments thereof with the understanding that the present disclosure is to 
be considered as an exemplification of the principles of the invention and is not intended to limit the invention to the 
specific embodiments illustrated. 

10 Figure 1 1llustrates an interactive learning system 10 which incorporates a plurality of components. The system 10 

includes a control unit 12, 12 are read-only memory 14 and read-write memory 16 which communicate with the proc- 
essor 12 via a bidirectional bus structure 16. The memories 14, 16. as is known in the art, can be used to permanently 
or temporarily store constants, programs being executed, or related data. 

Also coupled to the bus 1 8 Is a monitor, such as a CRT-type display 20, toud speakers 22, magnetic disk drive 24, 

IS CD-Rom Drive 26 and a user input device 28. The user input device 28 could be a keyboard, or alternatively a mouse. 
In addition, a kx^al area network (LAN) interface 30 enables the system 1 0 to communicate with the kx:al network. 
Further, a remote network interface 32 provkles bidlrectkxial communicatk)n with the Worldwide Web or other remote 
networks either through telecommunrcations or through other forms of data transmisston without limitation. 

In normal operation, programs can be stored on and read from disk drive 24 or CD-Rom drive 26. Graphk:s can 

20 also be read from the CD-Rom drive. Both the disk drive 24 and the CD-Rom drive 26 can be used as sources from 
whk;h databases can be accessed. 

During normal operatk>n of the system 10. visual information in lessons can be displayed on the display unit 20, 
under control of the processor 12. Text and graphics can be read from disk drive 24 or CD-Rom drive 26 for the purpose 
of displaying same to the user. As will be discussed subsequently, a control program for the purpose of carrying out 

2S the interactive learning process can be stored on one or both of disk drive 24 or CD-Rom 26 for access by processor 
12. Processor 12 is thus able to carry out a prestored process whk:h provides for indivklual assessment of a user's 
skills prior to the user going through the interactive learning process. 

Figure 2 is a flow diagram illustrating various steps of a miethod embodying the present Invention. In a step 100, 
the operating system for the system 10 is started. In a step 102 execution of the interactive learning system application 

30 program, of a type, for example, described In Lubin but for use with an icon-driven operating system such as available 
under the Trademark Windows 95 is started. In a step 104 the leaming system carries out an initial sequence of dis- 
playing titles and related text. When the system exits the initiating step 104, the user can select an introduction In a 
step 106 or can elect to turn directly to the course associated with the system in a step 1 08, 

In a step 1 1 0, a user can enter a course overview and pre-assessment testing sequence. After passing through a 

35 course overview in a step 112, the user enters the beginning of a plurality of pre-assessment sequences indicated 
generally at 114A, 114B...114N. The steps of a selected pre-assessment sequence are similar so a descriptk)n of the 
sequence 114a will also describe sequences 114B...114N. 

In a step 120. rnstructbns are presented on the display 20 for the user as to how to carry out the pre-assessment 
process. Subsequently, in a step 122 a plurality of questions or exercises are displayed, one at a time, In random order 

40 on the display 20. The user responds to each of the questions or exercises using the keyt>oard or mouse 28 and the 
responses are stored by the processor 12. In a step 124, the responses are compared to a pre-established standard, 
also stored in the disk drive 24 or the CD-Rom drive 26 to assess the performance of the user. For example and without 
limrtatbn, if questkxis are presented to the user, they can be ranged so as to have right or wrong answers or to have 
more preferred vs. less preferred answers whbh can be in tum reflected in the pre-stored standard. The correlatbn 

^ between the answers received from the user during the step 1 22 and the appropriate or desirable answers as reflected 
in the prestored standard, provides a basis for establishing a quantitative assessment of user performance. Subsequent 
to the step 124, if an individualized leaming profile is to be produced based on the sequence 114a, that profile is then 
produced in a step 126. Subsequent to the step 126, the leaming units can be presented to the user in a step 130 
discussed subsequently. 

so If desired, prbr to generation of the individualized learning profile in the step 126, additbnal pre-assessment se- 

quences 114B...114N can be executed to provkJe a composite learning profile. 

Figure 3 Illustrates in more detail the generation of the individualized leaming track as illustrated in the step 126. 
Test items 1-1 ...1-N to be presented in the step 122 are prestored in one of the disk drive 24 or CD-ROM drive 26. In 
the step 122a, one of the test questkxis or exercises is selected and presented to the user on the display 20. The 

ss answer or response received via the Input device 28 is compared to a previously-stored answer in a step 122b. If the 
answer is unsatisfactory, the objective associated with the test item and the associated topk: are retrieved in respective 
steps 122c. 122d. At least the associated topk: is then added to the individual leaming profile in a step 122e. If the 
questkxi is answered correctly in the step 122b, a determination Is made in the step 122f if the testing sequence is 
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finished. If it is not finished, the next test iteni is selected. If the test sequence is finished, either the next text sequence 
11 4b is entered or the teaming sequence associated with the individual profile is then executed in the step 1 30. 

Examples of test items follow as Examples 1 and 2. Example 1 illustrates two different matching-type test items. 
Example 2 Illustrates two different process educating test items. 

5 

Example 1 
Objective: 

10 Identify the Explorer features to view files and folders. 

Related Topic(s): 

Changing the Display 
IS Expanding and Collapsing Folders 

Test Item 1 : (question type = matching) 

Match the button to the task it performs: 

20 

a: ill Displays files as large icons 
b: ls| Displays file names in a list 
25 c: 1^ Displays file information for each file 
d: + Opens a folder 
e: J Closes a folder 

30 

Test Item 2: (question type = matching) 

Match the feature to the desired outcome. 

35 a: Arrange by Date Orders file names by last modification 

b: Arrange by Size Lists file names according to their space requirements 

c: Arrange by Type Displays all text documents 

d: Plus Sign Displays the files in a folder 

e: Minus Sign Closes an open folder containing files 

40 

Example 2 
Objective: 

^ Preview a file using Quick View on the sortcut menu. 

Related Toplc(s): 
Quick View 

so 

Test Item 1 : (question typessimulatlon) 

Complete the steps to see a Quick View of REPORT1 DOC using the shortcut menu. 
ss Test Item 2: (question type=simulation) 

Complete the actbn to preview a Quick View of Agenda doc using the shortcut menu. 

Figure 4 is a flow diagram illustrating in more detail executbn of the personal learning sequence, step 1 30. Where 
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the personal learning sequence, created in the step 122e includes a plurality of topics T^..T|^ the system 10 presents 
each of the selected topics in a series of steps 1 30-1 ...1 30-N to the user via the display 20. The user can interact with 
the various presented topics via the keytx)ard or mouse 28 during the process of going through that particular individ- 
ualized learning sequence. Subsequent to exiting the leaming sequence, the system 1 0 can present a post-assessment 

s test In a step 132 to detennine the level of skill that the user has improved to as a result of the process. A similar set 
of steps can be carried out to implement the personal leaming sequence for each of the units ..Uf^. 

Hence, when completed, the user will have experienced an individualized leaming sequence in connectkxi with 
one or more of the units 1 -N. The user can then exit to the step 109 and enter the application or other program being 
taught for direct execution or can enter additional course units for further instruction. 

10 Fig. 5 illustrates an overall flow diagram for an altemative approach to creating a personalized leaming experience 

based on collecting together a plurality of separately-accessible leaming objects. The leaming objects are contained 
In an accessible database which Is organized in several different ways, having both hierarchical and contact-sensitive 
aspects. 

A course object is made up of a plurality of independently-accessible unit objects. Unit objects are learning objects 
IS whkfh include a plurality of independent lesson objects. Lesson objects, in turn, are also learning objects which are 
made up of a plurality of dependent topic objects. Topic objects are dependent objects whk:h contain a single objective 
and an assessment. 

The content of a leaming object is illustrated, in an overall sense, in block 150 of Fig. 5. 

The lesson object is the smallest independent instructional activity available to a user. A given lesson object exhibits 
20 Intemai cohesion and is focused about a plurality of dependent topic objects. 

A lesson object incorporates a plurality of objectives as well as a second plurality of test or assessment items. It 
also incorporates predefined sequencing and interactk>ns between related, dependent, topic objects. 

Lesson objects can be linked with other lesson objects or can stand alone. Ibpic objects, by way of contrast, are 
dependent on the associated lesson object and contain only a single objective and an associated assessment. 
2S As illustrated in block 152 of Fig. 5, learning objects can be implemented out of a plurality of linked pages. The 

pages can be linked together both linearly and non-linearly. Pages can contain text, multi-media components and user 
interactk)n loge. 

Alternatively, leaming objects can be grouped together as a result of content or because one or more setves as 
a prerequisite for another, as illustrated in element 154 of Fig. 5. 

30 A plurality of leaming objects can be selected via a User through an object access program 156. The User can 

select, via the access program 156, one or more lesson objects to be experienced for the purpose of addressing an 
education or training objective. Leaming objects can be stored on a variety of media such as magnetic disc drives and/ 
or CD Rom drives. The object access program 156 can be accessed by the User not only directly through a kxal area 
network but also remotely via the Internet and the Wortdwide Web, if desired. 

35 From the foregoing, it will be obsen^ed that numerous variations and modlficatbns may be effected without de- 

parting from the spirit and scope of the invention. It is to be understood that no limitation with respect to the specific 
apparatus illustrated herein is intended or should be inferred. It is, of course, intended to cover by the appended claims 
all such modifications as fall within the scope of the claims. 

40 

Claims 

1 . A computer based method of automatically provkJing a paperiess, individualized, learning sequence comprising: 

^ storing in a computer readable database a plurality of questkxis directed toward assessing an individual's 

understanding of a selected learning objective for each member of a plurality of different learning objectives; 
automatically evaluating the individual by randomly and sequentially selected from the database at least one 
questk)n associated with each member of the plurality of leaming objectives and displaying each selected 
questbn to the individual for a response wherein each response is received and evaluated relative to a pre- 

so determined standard before the next question is selected; 

accumulating at least some of the results of the evaluatkxis in a computer storage unit in computer readable 
form; 

processing the results to automatically produce a specific sequence of instructk^al units in response to those 
responses which varied materially from the respective standard; and 

presenting the sequence of instructional units on a display for the individual wherein the indivktual can interact 
with each of the units via the display. 

2. A method as in claim 1 wherein in storing step, at least some of the questbns are linked to a plurality of possible 
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answers and in the displaying step, at least the selected question and the plurality of possible answers are dis- 
played. 

A method as in claim 1 wherein in the storing step, some of the questions are stored in a form which directs the 
individual to achieve a specified result without specifying the actions to be taken. 

A method as in claim 1 which includes presenting a performance evaluating test, subsequent to presenting the 
sequence of instructional units. 

An apparatus for providing an automatic, paperless, individualized learning sequence to a user comprising: 

a programmable processor; 

a visual display coupled to the processor; 

a unit coupled to the processor, for reading control information and a datat>ase from a medium and providing 
that information to the processor and wherein, in response to that information, the processor accesses the 
database and wherein the processor Includes circuitry for randomly selecting from the database a plurality of 
assessment questions and circuitry for presenting the plurality of questions to the user on the display; 
a user operable input device coupled to the processor, whereby the user provides responses to the processor 
to at least some of the assessment questions and wherein the processor includes further circuitry for evaluating 
the responses and In response thereto for establishing a sequence of selected topics for presentation to the 
user. 

A system as in claim 5 wherein the processor includes further circuitry for presenting the sequence of selected 
topics to the user. 

A system as in claim 6 wherein the processor includes further circuitry for reading from the database a post-test 
and circuitry for presenting the post test to the user. 

An individualized, paperless, learning system comprising: 

a database of different learning objects; 

a control unit with circuitry for enabling a user to select an individual sequence of learning objects from the 

database; 

a display unit coupled to the control unit whereat the control unit is adapted to display the individual sequence 
of leaning objects. 

A system as in claim 6 wherein the control unit includes circuitry for linking the objects in the individual sequence. 
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Fig. 2 
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Fig. 3 
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Fig. 4 



122d 



UNITIPLT 




UNrr2PLT 


■ • • 



130 



TOPIC 1 TO REVIEW J 



I 



130-1 



TOPIC 2 TO REVIEW 



rrOPICN TO REVIEW^ 



130^ 



UNITl POST- 
ASSESSMENT 



J 



UNITN 



10 



EP 0 838 798 A1 



Fig. 5 
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